Kernel Principal Components Are Maximum Entropy Projections
نویسندگان
چکیده
Principal Component Analysis (PCA) is a very well known statistical tool. Kernel PCA is a nonlinear extension to PCA based on the kernel paradigm. In this paper we characterize the projections found by Kernel PCA from a information theoretic perspective. We prove that Kernel PCA provides optimum entropy projections in the input space when the Gaussian kernel is used for the mapping and a sample estimate of Renyi’s entropy based on the Parzen window method is employed. The information theoretic interpretation motivates the choice and specifices the kernel used for the transformation to feature space.
منابع مشابه
Entropy in Multivariate Analysis: Projection Pursuit
Projection pursuit is an exploratory data-analytic method in multivariate (MV) analysis. It is similar to the well-known principal components analysis (PCA) in that it can be used to nd interesting structure within a MV data set. However, unlike PCA, which nds linear projections of maximum variance, projection pursuit nds linear projections of maximum non-normality, which sometimes is better at...
متن کاملKernel Based Subspace Projection of Near Infrared Hyperspectral Images of Maize Kernels
In this paper we present an exploratory analysis of hyperspectral 900-1700 nm images of maize kernels. The imaging device is a line scanning hyper spectral camera using a broadband NIR illumination. In order to explore the hyperspectral data we compare a series of subspace projection methods including principal component analysis and maximum autocorrelation factor analysis. The latter utilizes ...
متن کاملKernel Maximum Entropy Data Transformation and an Enhanced Spectral Clustering Algorithm
We propose a new kernel-based data transformation technique. It is founded on the principle of maximum entropy (MaxEnt) preservation, hence named kernel MaxEnt. The key measure is Renyi’s entropy estimated via Parzen windowing. We show that kernel MaxEnt is based on eigenvectors, and is in that sense similar to kernel PCA, but may produce strikingly different transformed data sets. An enhanced ...
متن کاملA Kernel PCA Method for Superior Word Sense Disambiguation
We introduce a new method for disambiguating word senses that exploits a nonlinear Kernel Principal Component Analysis (KPCA) technique to achieve accuracy superior to the best published individual models. We present empirical results demonstrating significantly better accuracy compared to the state-of-the-art achieved by either naı̈ve Bayes or maximum entropy models, on Senseval-2 data. We also...
متن کاملFault diagnosis of gearboxes using LSSVM and WPT
This paper concentrates on a new procedure which experimentally recognises gears and bearings faults of a typical gearbox system using a least square support vector machine (LSSVM). Two wavelet selection criteria Maximum Energy to Shannon Entropy ratio and Maximum Relative Wavelet Energy are used and compared to select an appropriate wavelet for feature extraction. The fault diagnosis method co...
متن کامل